Generally, people are often faced with performing tasks to solve problems in areas in which they are not experts. If there is a process with well-defined steps, a methodology, to attack a problem, then such a process is amenable to being represented as a recipe. FIG. 1A shows prior art technique for managing complex problems such as that presented by required network analysis of a test environment 99 on a rapid time scale. A manager 10 is responsible for collecting and organizing data information results and functions of application programs into meaningful results. In this example manager 10 is responsible for collecting and organizing production data 80 which is data collected during actual usage of test environment 99. Manager 10 is also responsible for collecting test data from various limited scope of operation test environments 30 running on system 20 with applications 25 to provide test data 40. Manager 10 is further responsible for invoking remote analysis program1 70, analysis program2 60, and analysis program3 50 to collect data, run low test scenarios and organize data into useful reports.
One example of analysis of a complex system is analysis of complex computer network, running one or more computer programs or applications forming a business system. The performance of business systems in the global economy is an area of considerable interest as businesses become more dispersed and applications become more complex. Decisions must be made rapidly and data systems must remain reliable and available. Reliability and performance can be a considerable issue in the face of rapid system or application scaling.
In this example of a complex system, a manager might be asked to optimize business applications on quite large and complex systems with perhaps many thousands of nodes that can be widely geographically dispersed. In order to meet this objective, a manager will design a test environment with actual equipment running actual business applications to be tested but on a much reduced scale from a production environment. The performance within the test environment is carefully measured and test data is recorded. The test data is used to project how the business application will perform in a more complex production or projected environment. In other situations, a system may be overly stressed, with such low business application performance that the situation is detrimental to the function of the corporation; the manager is often asked to troubleshoot the problem quickly. To accommodate the manager a methodology or series of steps must be adopted to complete the various tasks required to acquire and manipulate the test data to achieve meaningful results. Furthermore, rapidly visualizing the methodology optimizes the ability to draw conclusions and make decisions. Moreover a system which allows the methodology to be captured and reused in other situations or on other networks further enhances the usefulness of the manager.
FIG. 1B shows an example of a test environment 100 to investigate application and network performance. In the example, is a network of servers, workstations, business applications, data storage devices, test devices and IP network connections between them shown as LAN 114 and Internet 105. More specifically, the network of servers is comprised of application server 124 connected to LAN 114 which runs business, engineering or research applications, database server 120 connected to LAN 114 and which is a local database that organizes information of interest to the business, storage server 135 connected to LAN 114 and which holds a local data storage 138 that feeds the servers and to which data is backed up from the servers, remote database server 190 connected to LAN 114 via the Internet 105 and remote LAN 117 and which is geographically remote from database server 120 and serves a similar function to the local server but may house different pieces of information from different business units, and remote storage server 150 connected to LAN 114 via the Internet 105 and remote LAN 116 and which is used to keep a synchronous or asynchronous copy of the local data storage 138 to remote data storage 155. A workstation 130 is shown which runs a first application client 101 and a second application client 102; workstation 130 is also connected to LAN 114. Interspersed between the LAN 114 and the various servers are network sniffer devices 140, 144, 150 and 160. There is a network sniffer device 170 between LAN 114 and the internet 105. Network sniffer devices 175 and 185, are respectively connected between the remote data storage 155 and remote storage server 150 and between the local data storage 138 and the local data storage server 135. There is also a network sniffer device 180 between the internet 105 and remote database server 190. The network sniffers function to examine data packets as they go through the network looking for a match and logging a timestamp for each match. They will also count the number of packets that match in a given time frame and perform other such functions related to network packet timing.
The various devices shown in FIG. 1B can be logically connected to one or more analysis programs to record and organize network data. For example, analysis program1 191, is shown logically connected to storage server 135, data base server 120, application server 124 and workstation 130. Analysis programs may be configured to capture, for example, research usage of the various components of the network.
Another example is analysis program2 192 shown logically connected to sniffers 160, 140, 144 and 150. This analysis program may be configured, for example, to capture network trace data to establish flow of business process through the network. Analysis program3 193 is shown connected to internet 105 and may be configured to analyze network communications data from related components.
Another example is analysis program4 194 shown logically connected to remote storage server 150, remote data storage 155 and storage server 135. Analysis program4 may be configured, for example, to analyze the performance of each vehicle for data storage.
More generally, users run across many difficulties in following a complex methodology that are solved by standard project software, including                Tracking progress over weeks or months (solved with done flags and automatic journal logging), Organizing complexity (solved with hierarchy),        Understanding intermediate and longer desirable goals (solved with complete and random access to recipe),        Understanding how to manipulate many individual tools (solved by individual step kinds), Customizing a recipe for some specific use (solved by recipe editing capabilities),        Knowing the effects of iterating again through a process (solved by dynamic configuration) Reusing recipes on different problems (solved by using recipe templates).        Providing data traceability by allowing users to take a result and trace backwards through all the documents and calculations to the original data, being able to identify problems in the process that created the result. (solved by unfolding lists of steps with clearly identified dependencies describing inputs)        
Obviously, a complex problem presents virtually infinite number of possibilities of ordering of steps in order to accomplish analysis of the complex process. Therefore a need exists to adopt a methodology where recipes can be used to organize and streamline the process of data collection and analysis of a complex system. The different problem domains where recipes may be applicable is varied, even though the present invention is initially targeted at data analysis.
Until the invention described herein, the processes above were performed primarily through time-consuming and error-prone manual data manipulation by skilled performance engineers. This invention automates much of this process, thereby reducing the time required and improving the accuracy in solving complex problems.